System and method for rendering virtual reality interactions

ABSTRACT

A Virtual Reality (VR) computer system and method including a VR headset to be worn by at least one user; one or more pupil sensors located in the VR headset configured and operative to capture and track pupil movement of the at least one user wearing the VR headset; and at least one camera device operative to capture video image of at least one user wearing the VR headset. A computer processor is instructed to generate a two-dimensional (2D) image of the least one user such that an image of the VR headset is virtually removed from the image of the at least one user wearing the VR headset.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Patent Application Ser. No.62/551,960 filed Aug. 30, 2017 which is incorporated herein by referencein its entirety.

FIELD OF THE INVENTION

This description generally relates to methods and devices for capturingand processing two-dimensional (2D) and three-dimensional (3D) images,and more particularly to virtually removing user headsets during a VRinteraction between two or more users

BACKGROUND OF THE INVENTION

Virtual Reality (VR) enables remarkably immersive experiences, offeringnew ways to view the world and the ability to explore novelenvironments, both real and imaginary. However, compared to physicalreality, sharing these experiences with others can be difficult, as VRheadsets make it challenging to create a complete picture of the peopleparticipating in the experience.

Some of this disconnect is alleviated by Mixed Reality (MR), a relatedmedium that shares the virtual context of a VR user in a two dimensionalvideo format allowing other viewers to get a feel for the user's virtualexperience. Even though MR facilitates sharing, the headset continues toblock facial expressions and eye gaze, presenting a significant hurdleto a fully engaging experience and complete view of the person in VR.

SUMMARY OF THE INVENTION

The purpose and advantages of the below described illustratedembodiments will be set forth in and apparent from the description thatfollows. Additional advantages of the illustrated embodiments will berealized and attained by the devices, systems and methods particularlypointed out in the written description and claims hereof, as well asfrom the appended drawings.

The general purpose of this project is to provide a live VR chatexperience. Currently VR chat applications make use of animated avatars.An object of the present invention is to replace the avatars with livevideo without the image of a user's headsets which would be replacedvisually by a 3D model of the users' head (e.g., recorded previouslywithout the headset being worn by a user). Another object of the presentinvention is each user may adjust their level of realism (e.g., fromavatar to live 3D video image).

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying appendices and/or drawings illustrate variousnon-limiting, example, inventive aspects in accordance with the presentdisclosure:

FIG. 1 illustrates an example of a computer device and a mobile devicethat can be used to implement the techniques described herein;

FIG. 2 depicts users wearing headsets and the view a headset wearinguser perceives of another headset wearing user;

FIG. 3 depicts the process for removing the image of a headset from auser to create a VR of the user as if the headset was removed; and

FIGS. 4-7 illustrate portions of the process for creating a VR chatbetween users in accordance with the described illustrated embodiments.

DETAILED DESCRIPTION OF CERTAIN EMBODIMENTS

The illustrated embodiments are now described more fully with referenceto the accompanying drawings wherein like reference numerals identifysimilar structural/functional features. The illustrated embodiments arenot limited in any way to what is illustrated as the illustratedembodiments described below are merely exemplary, which can be embodiedin various forms, as appreciated by one skilled in the art. Therefore,it is to be understood that any structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as abasis for the claims and as a representation for teaching one skilled inthe art to variously employ the discussed embodiments. Furthermore, theterms and phrases used herein are not intended to be limiting but ratherto provide an understandable description of the illustrated embodiments.

Where a range of values is provided, it is understood that eachintervening value, to the tenth of the unit of the lower limit unlessthe context clearly dictates otherwise, between the upper and lowerlimit of that range and any other stated or intervening value in thatstated range is encompassed within the illustrated embodiments. Theupper and lower limits of these smaller ranges may independently beincluded in the smaller ranges is also encompassed within theillustrated embodiments, subject to any specifically excluded limit inthe stated range. Where the stated range includes one or both of thelimits, ranges excluding either both of those included limits are alsoincluded in the illustrated embodiments.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although any methods andmaterials similar or equivalent to those described herein can also beused in the practice or testing of the illustrated embodiments,exemplary methods and materials are now described. All publicationsmentioned herein are incorporated herein by reference to disclose anddescribe the methods and/or materials in connection with which thepublications are cited.

It must be noted that as used herein and in the appended claims, thesingular forms “a”, “an,” and “the” include plural referents unless thecontext clearly dictates otherwise. Thus, for example, reference to “astimulus” includes a plurality of such stimuli and reference to “thesignal” includes reference to one or more signals and equivalentsthereof known to those skilled in the art, and so forth.

The publications discussed herein are provided solely for theirdisclosure prior to the filing date of the present application. Nothingherein is to be construed as an admission that the illustratedembodiments are not entitled to antedate such publication by virtue ofprior invention. Further, the dates of publication provided may differfrom the actual publication dates which may need to be independentlyconfirmed.

It is to be appreciated the illustrated embodiments discussed below arepreferably a software algorithm, program or code residing on computeruseable medium having control logic for enabling execution on a machinehaving a computer processor. The machine typically includes memorystorage configured to provide output from execution of the computeralgorithm or program.

As used herein, the term “software” is meant to be synonymous with anycode or program that can be in a processor of a host computer,regardless of whether the implementation is in hardware, firmware or asa software computer product available on a disc, a memory storagedevice, or for download from a remote machine. The embodiments describedherein include such software to implement the equations, relationshipsand algorithms described above. One skilled in the art will appreciatefurther features and advantages of the illustrated embodiments based onthe above-described embodiments. Accordingly, the illustratedembodiments are not to be limited by what has been particularly shownand described, except as indicated by the appended claims. Allpublications and references cited herein are expressly incorporatedherein by reference in their entirety.

Turning now descriptively to the drawings, in which similar referencecharacters denote similar elements throughout the several views, FIG. 1depicts an exemplary general-purpose computing system 600 and a genericmobile computer device 650, which may be used with the techniquesdescribed here. It is to be appreciated computing system 600 and mobiledevice 650 are to be understood as an exemplary embodiments in which oneor more components thereof may be used with the illustrated embodimentsof the present invention as described herein.

Computing device 600 includes a processor 602, memory 604, a storagedevice 606, a high-speed interface 608 connecting to memory 604 andhigh-speed expansion ports 610, and a low speed interface 612 connectingto low speed bus 614 and storage device 606. Each of the components 602,604, 606, 608, 610, and 612, are interconnected using various busses,and may be mounted on a common motherboard or in other manners asappropriate. The processor 602 can process instructions for executionwithin the computing device 600, including instructions stored in thememory 604 or on the storage device 606 to display graphical informationfor a GUI on an external input/output device, such as display 616coupled to high speed interface 608. In other implementations, multipleprocessors and/or multiple buses may be used, as appropriate, along withmultiple memories and types of memory. In addition, multiple computingdevices 600 may be connected, with each device providing portions of thenecessary operations (e.g., as a server bank, a group of blade servers,or a multi-processor system).

The memory 604 stores information within the computing device 600. Inone implementation, the memory 604 is a volatile memory unit or units.In another implementation, the memory 604 is a non-volatile memory unitor units. The memory 604 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 606 is capable of providing mass storage for thecomputing device 600. In one implementation, the storage device 606 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 604, the storage device 606,or memory on processor 602.

The high speed controller 608 manages bandwidth-intensive operations forthe computing device 600, while the low speed controller 612 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 608 iscoupled to memory 604, display 616 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 610, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 612 is coupled to storage device 606 and low-speed expansionport 614. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 600 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 620, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 624. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 622. Alternatively, components from computing device 600 may becombined with other components in a mobile device (not shown), such asdevice 650. Each of such devices may contain one or more of computingdevice 600, 650, and an entire system may be made up of multiplecomputing devices 600, 650 communicating with each other.

Computing device 650 includes a processor 652, memory 664, aninput/output device such as a display 654, a communication interface666, and a transceiver 668, among other components. The device 650 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 650, 652,664, 654, 666, and 668, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 652 can execute instructions within the computing device650, including instructions stored in the memory 664. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 650, such ascontrol of user interfaces, applications run by device 650, and wirelesscommunication by device 650.

Processor 652 may communicate with a user through control interface 658and display interface 656 coupled to a display 654. The display 654 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display)or an OLED (Organic Light Emitting Diode) display, or other appropriatedisplay technology. The display interface 656 may comprise appropriatecircuitry for driving the display 654 to present graphical and otherinformation to a user. The control interface 658 may receive commandsfrom a user and convert them for submission to the processor 652. Inaddition, an external interface 662 may be provide in communication withprocessor 652, so as to enable near area communication of device 650with other devices. External interface 662 may provide, for example, forwired communication in some implementations, or for wirelesscommunication in other implementations, and multiple interfaces may alsobe used.

The memory 664 stores information within the computing device 650. Thememory 664 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 674 may also be provided andconnected to device 650 through expansion interface 672, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 674 may provide extra storage space fordevice 650, or may also store applications or other information fordevice 650. Specifically, expansion memory 674 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 674may be provide as a security module for device 650, and may beprogrammed with instructions that permit secure use of device 650. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 664, expansionmemory 674, or memory on processor 652, that may be received, forexample, over transceiver 668 or external interface 662.

Device 650 may communicate wirelessly through communication interface666, which may include digital signal processing circuitry wherenecessary. Communication interface 666 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 668. In addition, short-range communication may occur, suchas using a Bluetooth, Wi-Fi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 670 mayprovide additional navigation- and location-related wireless data todevice 650, which may be used as appropriate by applications running ondevice 650.

Device 650 may also communicate audibly using audio codec 660, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 660 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 650. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 650.

The computing device 650 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 680. It may also be implemented as part of a smartphone 682, personal digital assistant, or other similar mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)(and other suitable display technology) for displaying information tothe user and a keyboard and a pointing device (e.g., a mouse or atrackball) by which the user can provide input to the computer. Otherkinds of devices can be used to provide for interaction with a user aswell; for example, feedback provided to the user can be any form ofsensory feedback (e.g., visual feedback, auditory feedback, or tactilefeedback); and input from the user can be received in any form,including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

The computing devices depicted in FIG. 1 interface with a virtualreality (VR headset 690). In an illustrated embodiment, one or moresensors are included in the VR headset to virtual remove the appearanceof the headset 690 when one user interacts with another user. Forinstance, the headset sensors 690 may include one or more LED's 691 thatilluminates an eye of a user while tracking the user's pupil and one ormore micro cameras 693 for recording the user's eye/pupil.

For instance, an embodiment of the present invention creates a model ofthe user's face 704 without a headset 690 using a camera device 400 as aproxy for the user's hidden face when a headset 690 is worn. This proxyis used to synthesize the face in a Mixed Reality (MR) video, therebycreating an impression of the headset 690 being removed when one usinginteracts with another user in a VR realm. For instance, this initialcalibration step may require the user 704 to sit in front of acolor+depth camera 400 and a monitor, and then track a marker on themonitor with their eyes. This calibration procedure acquires a 3D facemodel of the user 704, and create a database that maps appearance images(or textures) to different eye-gaze directions and blinks (via sensors691 and 693). The aforementioned eye-gaze database (i.e. the face modelpreferably with textures indexed by eye-gaze) enables the appearance ofa user's face to be dynamically changed to generate desired (andreal-time) eye-gaze, thus making the synthesized face look natural andalive. It is to be appreciated the construct of the face can also bealtered by using suitable wire frame software technology.

In some implementations, one or more input devices in addition to thecomputing device (e.g., a mouse, a keyboard) can be rendered in acomputer-generated, 3D environment. The rendered input devices (e.g.,the rendered mouse, the rendered keyboard) can be used as rendered inthe VR space to control objects in the VR space.

Computing device 600 is intended to represent various forms of digitalcomputers, such as laptops, desktops, workstations, personal digitalassistants, servers, blade servers, mainframes, and other appropriatecomputers. Computing device 650 is intended to represent various formsof mobile devices, such as personal digital assistants, cellulartelephones, smart phones, and other similar computing devices. Thecomponents shown here, their connections and relationships, and theirfunctions, are meant to be exemplary only, and are not meant to limitimplementations of the inventions described and/or claimed in thisdocument.

With the exemplary computing system environment 100 of FIG. 1 beinggenerally shown and discussed above, and with reference now to FIGS.2-7, described below is implementation of various exemplary embodiments.

It is to be appreciated an environment of use for an illustratedembodiment of the present invention relates to a system 600 that enablesonline chat in VR preferably with live video. In the example discussedbelow used to described an illustrated embodiment, it is to beunderstood it assumes a two person live chat with a local user and aremote user (which the present invention is not to be understood to belimited to). Exemplary embodiments include one or more components of theabove described system 600 including a VR system such as Oculus Rift™,one or more webcams 400 per user and preferably a green screen. Withreference to FIG. 2, during a VR chat session in accordance with anillustrated embodiment, a remote user 700 wearing headset 690 forexample would view a live video of another user 702 (wearing headset690), minus the headset 690 that the local user is wearing 704, in VR.Likewise, during the same VR chat session, the remote user 702 wearingheadset 690 would view a live video of the user 700 (wearing headset690), minus the headset 690 that the local user is wearing 706, in VR.

In accordance with an aspect of the present invention (and withreference to FIG. 3), the headset visual for a remote user is replacedby an overlay of pre-recorded static images/video of the local users'head. Preferably only a portion of the users head needs to besuperimposed, as the mouth and lower portion of the face is stillgenerally visible when wearing VR headsets. Preferably, the live videoeffect is created by chromakey (“green screen”) technology preferablyusing a standard 2D webcam 400 (FIG. 4). Thus, in accordance with anillustrated embodiment, each user 704, 706 is preferably placed in frontof a portable green screen (402). It is to be appreciated this may benegated by certain software techniques or alternatively it may bepossible to determine the users outline without a green screen by usinga 3D camera. As described below, in accordance with an illustratedembodiment, two or more users (704 and 706) may participate in a liveonline chat using a special webcam 400 at each user location. Each user(704, 706) preferably wears a VR headset 690 (700, 702) whereby eachuser (704, 706) views a superimposed and prerecorded image of the otheruser's head (706, 704). The broadcast image is preferably a composite ofa live video and prerecorded image. The live video preferably includesthe body up to just above a user's mouth region (FIG. 3) (e.g., wherethe VR headset 690 generally resides). Thus, the recorded image ispreferably the nose, eyes and top of head of a user (704, 706). It is tobe appreciated that it should be possible to achieve a live chat withmore than two persons, such as a corporate teleconference.

This recorded image may also be created prior using a special form ofphotogrammetry. This is the technique where multiple 2D images arestitched together and processed to create a 3D model and texture.Currently this process is expensive, time consuming, and requiresspecial hardware and multiple cameras. In contrast, in accordance withan embodiment of the present invention, implementation of inexpensivestereoscopic cameras may be utilized consisting of two cameras facingthe user, each with either stereoscopic or monoscopic vision. Forinstance, the user would rotate in their chair while the cameras recordand capture the scene. Post processing of the stereoscopic or monoscopiccamera output would be used to generate the 3D model.

With reference now to FIGS. 4-7, an exemplary process for enabling a VRsession in accordance with an illustrated embodiment is described. Firstthe face of each user 704 (FIG. 4), 706 (FIG. 5) without a headset isphotographed by webcam 400. As shown in FIGS. 6 and 7, each user (704,706) then wears a headset 690 having the aforementioned LED sensors 691and 693 for detecting eye gaze, movement and blinking. Preferably thesensors 691 illuminates one eye of the user inside the headset 690 totrack pupil movement while the micro video camera sensor 693 records auser's pupil. The processing chip 602 is preferably configured andoperational to restore the moving pupil to the face by “stitching” it inpreferably via software to complete the image of the face currentlyobscured by the headset 690. Thus, both users 704 and 706 can view 3Dimages of each other (via their respective headset 690) in real time asif the face of the other user 704, 706 were in front of them andunobscured by a headset 690.

It is to be further appreciated, that through the further use ofelectro-sensitive devices positioned preferably on the forehead andeyebrows of each user, the frowns expression of each user 704, 706 mayalso be tracked via software processing on the processing chip 602.

In summary, and in accordance with one or more illustrated embodiments,the present invention enables virtual projection of a webcam videostream of a user in a video chat, minus the users' headset. This isaccomplished by modeling the users head in 3D using modellingtechniques, such as photogrammetry. For instance, the top portion of ausers' head is replaced by this model in real time, so the otherparticipating user only views the body, head, face and eyes of aparticipating user, minus the headset 690.

It is to be appreciated multiple users (e.g., greater than two users)are to be encompassed wherein each user is modelled using one or morestationary stereoscopic 3D cameras (e.g., StereoLabs ZED camera,Microsoft Kinect, Intel RealSense line of 3D cameras, etc.). The 3Dcamera techniques additional may calculate a user's position, providingVR cooperative interaction. For instance, two users could play a game ofcatch on Mars or two people could take a virtual walk together in theGrand Canyon.

It is to be appreciated the present invention computing system 600 andheadset 690 may be configured and operative for use in an AugmentedReality (AR) setting. AR is to be understood to be a technology thatlayers computer-generated enhancements atop an existing reality in orderto make it more meaningful through the ability to interact with it. Anillustrated embodiment includes AR being developed for use as an apppreferably a the headset 690 to blend digital components into the realworld in such a way that they enhance one another. For instance, two ormore users, each in a different location, wearing the present inventionheadset 690 may use/operate the headsets 690 so as to appear next to oneanother in a chosen setting (e.g., skydiving) and/or location (e.g.,Paris) whereby they not only in real-time visualize the same settingtogether (e.g., a street in Paris, France) but they view each other asif the headset 690 were removed, in accordance with the teachingsherein.

With certain illustrated embodiments described above, it is to beappreciated that various non-limiting embodiments described herein maybe used separately, combined or selectively combined for specificapplications. Further, some of the various features of the abovenon-limiting embodiments may be used without the corresponding use ofother described features. The description provided herein shouldtherefore be considered as merely illustrative of the principles,teachings and exemplary embodiments of this invention, and not inlimitation thereof.

What is claimed is:
 1. A method for creating Virtual Reality (VR)interactions between two or more users wherein at least one user iswearing a VR headset, comprising: generating a two-dimensional (2D)image of at least one user wearing a VR headset using a camera devicewherein positioned in the headset is one or more sensors that detect andrecord pupil images; tracking the pupil movement of the user using theone or more sensors; removing a headset image from the generared 2Dimage of the user wearing the headset; and stitching in the detected andrecorded pupil image of the at least one user so as to be superimposedin an area proximate to where the headset image was removed from the atleast one user such that the generated 2D image of the at least one canbe viewed by another user whereby an image of the VR headset worn by theat least one user is removed and replaced by the at least one user'spupils.
 2. The method as recited in claim 1, wherein the 2D image iscreated using a webcam.
 3. The method as recited in claim 1, wherein theone or more sensors include infrared LED sensors.
 4. The method asrecited in claim 3, wherein the one or more sensors further includemicro video camera for recording aspects of a user's pupil.
 5. Themethod as recited in claim 1, wherein user's may view each other inreal-time.
 6. A computer-readable storage medium accessed by a computersystem for providing instructions and data to the computer system thatcause the computer system to perform the following: receive from acamera device an image of at least one user wearing a VR headset;utlizing one or more sensors located in the VR headset, capture andtrack pupil movement of the at least one user wearing the VR headset;generate a two-dimensional (2D) image of the least one user using thecamera device such that an image of the VR headset is virtually removedfrom the image of the at least one user wearing the VR headset; andstitching in the generated 2D image a detected and recorded pupil imageof the at least one user captued by the one or more sensors such thatthe at least one user's pupil image is superimposed proximate to thelocation the headset image was virtually removed from the generated 2Dimage of the at least one user such that another user can view a VRimage of the at least one user having pupil images superimposed in thearea the headset image was virtually removed.
 7. The computer system asrecited in claim 6, wherein the camera device is a webcam.
 8. Thecomputer system as recited in claim 6, wherein the one or more sensorslocated in the VR headset include infrared LED sensors.
 9. The computersystem as recited in claim 8, wherein the one or more sensors furtherinclude a video camera for capturing video image of the user's pupil.10. The computer system as recited in claim 6, wherein the generated 2Dimage of the user having the superimposed pupil image and movement isrendered in real-time.
 11. A Virtual Reality (VR) computer system,comprising: a VR headset to be worn by at least one user; one or morepupil sensors located in the VR headset configured and operative tocapture and track pupil movement of the at least one user wearing the VRheadset; at least one camera device operative to capture video image ofat least one user wearing the VR headset; a computer processor havinginstrutions to: receive from the camera device a video image of the atleast one user wearing the VR headset; utlizing the one or more pupilsensors, capture and track pupil movement of the at least one userwearing the VR headset; generate a two-dimensional (2D) image of theleast one user such that an image of the VR headset is virtually removedfrom the image of the at least one user wearing the VR headset; andstitching in the generated 2D image detected and recorded pupil image ofthe user captued by the one or more sensors such that the at least oneuser's pupil image is superimposed proximate to the location the headsetimage was virtually removed from the generated 2D image of the at leastuser such that another user can view a VR image of the at least one userhaving the superimposed pupil image and movement such that the at leastone user was not wearing the VR headset.
 12. The VR system as recited inclaim 11, wherein the camera device is a webcam.
 13. The VR system asrecited in claim 11, wherein the one or more sensors located in the VRheadset include infrared LED sensors.
 14. The VR system as recited inclaim 13, wherein the one or more sensors further include a video camerafor capturing video image of the user's pupil.
 15. The VR system asrecited in claim 11, wherein the generated 2D image of the user havingthe superimposed pupil image and movement is rendered in real-time.